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Abstract — We present a multidimensional evidential reasoning 
(MDER) approach to estimate change detection from the fusion 
of heterogeneous remote sensing images. MDER is based on a 
multidimensional (M-D) frame of discernment composed by the 
Cartesian product of the separate frames of discernment used 
for the classification of each image. Every element of the M-D 
frame is a basic joint state that allows to describe precisely the 
possible change occurrences between the heterogeneous images. 
Two kinds of rules of combination are proposed for working 
either with the free model, or with a constrained model depending 
on the integrity constraints one wants to take into account in the 
scenario under study. We show the potential interest of the MDER 
approach for detecting changes due to a flood in the Gloucester 
area in the U.K. from two real ERS and SPOT images. 

Index Terms — Belief functions, change detection, Dezert- 
Smarandache Theory (DSmT), Dempster-Shafer Theory (DST), 
remote sensing (RS). 

I. Introduction 

I N CHANGE detection from heterogeneous remote sensing 
(RS) images, many works have been devoted to change 
measure [1]— [5]. Recent works have also been done on the 
classification of changed features [6]— [10] . Particularly, an 
unsupervised change detection approach has been proposed 
in [4] for dealing with multisource and multisensor remote 
sensing images that allows to integrate the estimates of sta- 
tistical terms achieved on the difference images. In [5], the 
similarity between the predicted image obtained from optical 
image and the SAR image is used to detect the damages 
caused by an earthquake. The heterogeneous remote sensing 
images are usually acquired from different kinds of sensors. 
Therefore, the classification does not necessarily involve the 
same classes definition (i.e., surface roughness from radar 
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versus chlorophyll from optical sensor, etc.). Moreover, the 
images usually include uncertain information due to noises, 
and imprecise information due to lack of knowledge specially 
at the transition between areas [11]. The limited quality of 
information makes the detection of changes between hetero- 
geneous remote sensing images difficult to accomplish. In this 
letter, the change detection is considered as a classification 
problem, and we envisage the fusion of the classified images as 
appropriate for the change detection in heterogeneous remote 
sensing images. The fusion of classified images requires 
efficient tools for working with uncertain, imprecise, and 
even conflicting pieces of information. The theories of Evi- 
dences, including the Dempster-Shafer Theory (DST) [12] and 
Dezert-Smarandache Theory (DSmT) [13] propose theoretical 
frameworks to deal with uncertain and imprecise information. 
These theories have already been applied for the fusion of 
remote sensing images [14], [15]. 

The classical DST framework is not well adapted for change 
detection between images. Indeed, all the conflicting masses 
of belief that could be used to detect the changes are added 
altogether in a total degree of conflict that enters in the 
normalization constant of Dempster-Shafer (DS) fusion rule. 
To overcome the flaws and limitations of DST and DS rule 
[16], [17], new possible modelings of the frame of discernment 
and rules of combination were proposed in DSmT [13] for 
dealing separately with all the partial conflicts. Unfortunately, 
for change detection in RS images, the partial conflicting 
elements do not model sufficiently well the change occurrences 
since the element A n B = B n A cannot distinguish the 
change occurrences from A to B or from B to A. In our 
previous works [18], [19], a dynamic evidential reasoning 
(DER) approach had been developed for the change detection 
from RS images issued from the same type of remote sensor 
(e.g., a pair of SPOT images). In DER approach, the classes of 
these images were elements of the same frame of discernment. 
In this letter, we mainly focus on the change detection from 
the heterogeneous remote sensing images (e.g., from optical 
and radar images), where the classes of the images can be 
defined in distinct frames of discernments. 

We propose a general multidimensional evidential reason- 
ing (MDER) approach, which is effective for the change 
detection from both heterogeneous and homogenous re- 
mote sensing images. MDER is designed for the fu- 
sion of multisources classified images using same or dis- 
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tinct frames of discernment depending on the classifica- 
tions done on the images. MDER can be considered as an 
extension of classical evidential reasoning methods (DST, 
DSmT, and DER) for working under the multidimensional 
(M-D) 1 frame of discernment composed by the Cartesian 
product of the separate monodimensional (1-D) frames. 2 The 
elements of the M-D frame are called basic joint states. The 
space of the fusion result lies in the M-D power- set, which is 
constructed by the basic joint states with the union operator. 
The number of dimensions of the basic joint state depends 
on the number of images available. The joint states offer 
a better representation of the image mapping and change 
occurrences in the fusion of different images. The MDER 
approach provides a more refined information than what we 
get from the classical evidential reasoning approaches when 
working only in a 1-D frame. For dealing with different 
situations encountered in real applications, we provide two 
rules of combination adapted for the free and the constrained 
models of the M-D frame of discernment. 

Section II details the framework of MDER, while Section III 
focuses on the combination rules that allows the classification 
of heterogeneous data, as well as the change detection from a 
set of heterogeneous data. Section IV presents an application 
of MDER using real heterogeneous data (i.e., SPOT and ERS 
images). Conclusions and perspectives are given in Section V. 

II. Multidimensional Evidential 
Reasoning Framework 

A. Space of Multidimensional Evidential Reasoning 

In the MDER approach, the fusion space is always increas- 
ing with the number of sources, no matter if the 1-D frames of 
the sources to combine are identical or not. Even if the sources 
of evidence obtained from each image are respectively defined 
in 1-D frames, their fusion results will be considered in M-D 
frame defined by the Cartesian product of these 1-D frames to 
better represent the joint state of the heterogeneous images. 

Let us consider n sources of evidence respectively de- 
fined with respect to n 1-D frames ©i, © 2 , , ©„. The 

frame of classifications of the /th image (/ = 1, ... ,n) is 
©/ = bi- 1 , T/; 2, • • • , yi-jiih where elements y^, k = 1 
are the classes available in the /th image. The frames ©; 
and © y -, i f j can be similar, or different depending on 
the classifications done in each image. The M-D frame is 
= ©! x ©2 x • • • x ©„, where symbols x denotes the 
Cartesian product operator, and = means equals by definition. 
The cardinality of ^ n is \^ n \ = |©i | x |© 2 | x • • • x |©„ |, where 
\X\ being the cardinality of X is the number of singletons in 
X. The M-D element (yi, y 2 , • • • 9 y n ) is called the basic 
joint- state of different images, with the interpretation that in 
the same region of n sources of coregistered multitemporal 
images, the content of this region in the image number i 
belongs to yi , for i = 1, . . . , n. The fusion space of MDER 
is given by the power set of the product frame, which is 

1 M-D means that the elements are obtained from multiple frames of 
discernment. 

2 1-D here means that the elements are all from the same unique frame of 
discernment 



2^ n = 2 0 i x0 2 x " x0 «. The M-D power-set 2^" is composed by 
all the subsets of ^ n . The cardinality of is |2^”| = 2 |VI '" 1 = 
2 1 0 1 1 x 1 0 2 1 x * ■* x 1 0 n | ^ j n MDER approach, the operators U and H 
applied to elements A, B e 2 01 , and C, D e 2 02 must satisfy 
the following conditions: 

Cl) Componentwise distributivity of union: 

(A U B, C U D) = (A, C) U (A, D) U ( B , C) U ( B , D) 
C2) Componentwise intersection: 

(A, c) n ( b , D) = (A n b, c n d) 

C3) Vacuity of joint states: (A, 0) = (0, A) = 0. 

B. Basic Definitions 

In the MDER approach, a basic belief assignment (bba) is 
defined as a function ra(-) from the M-D power-set 2^" to 
[0, 1], verifying m(0) = 0 and 

Y j m{A)=\. (1) 

Ae 2*" 

Any element A e 2^” such that m(A) > 0 is called a M-D 
focal element of m(-). All the imprecise joint states can be 
decomposed in the canonical disjunctive form using the basic 
joint states with the operator U according to the condition 
Cl. For example, (X U Y, Z) = (X, Z ) U (Y, Z). With the 
canonical disjunctive form of the joint states, the belief Bel(-) 
and plausibility Pl(-) functions are defined by 



Bel(A) = 


m(B) 


(2) 




A,Be2^ n \BcA 




P1(A) = 


Y m ^- 


(3) 




A,Be2^ n |An#/0 





The interval [Bel(A), P1(A)] is classically interpreted [12] 
as the lower and upper bounds of imprecise probability for 
decision-making support. The pignistic probability BetP(A) 
[20] commonly used to approximate the unknown probability 
in [Bel(A), P1(A)] is calculated by 

\A n B\ 

BetP(A) = Y, (4) 

Be2^ n ,B^& 

In MDER, the cardinality of A e 2^" is the number of the 
basic joint states contained in the canonical disjunctive form 
of A. 

Example 1: Let us consider the following bba’s over = 
© x £2 with © = {#1 , 62} and £2 = {co\, 002, 003} : m(6 \ , C02) = 
0.5, m( 0 \, CO2UCO3) = 0.2 and m(©, (03) = 0.3. The cardinality 
of the imprecise joint states is counted as follows: 

l( 0 i, ^2 U 0)3)1 = \(0 u a) 2 )U(0 u a)3)\ =2 

l(©, <*>3)\ = 1(01 U 02, &>3)l = 1(01, ^ 3 ) U (02, Cl>3)\ = 2 
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Then one gets Bel(0i, W 3 ) = Bel(02, w 3 ) = 0 and 



Bel(0i, ( 02 ) = m(0\, ( 02 ) = 0.5 
Pl(0i , CO 2 ) = m(0\ , CO 2 ) + m(0\ , (jl >2 U 0 ) 3 ) = 0.7 
PI (01 , C 03 ) = m( 6 \ , ct >2 U 00 3 ) + m(0 , 0 ) 3 ) = 0.5 
PI (02, (*> 3 ) = m( 0 , 00 3 ) = 0.3 



BetP(0i, 002 ) = m(0i , 002 ) + 



BetP(0i , o) 3 ) = 



m(0i, 002 U o) 3 ) 



m(0\, 0)2 U 0 ) 3 ) 
2 

m(0, a; 3 ) 



= 0.6 



= 0.25 



777 ( 0 , 073) 

BetP(0 2 , 073 ) = — ^ = 0.15. 



III. Combination Rules in Multidimensional 
Evidential Reasoning 

In DST [12] or in DSmT [13], the source of evidence pro- 
vide their bba’s defined on a same frame of discernment, and a 
particular attention is focused on the way in which the conflict- 
ing masses of belief are redistributed. In the DER approach 
[18], the sources of evidences are also defined on the same 
frame of discernment and they are sequentially combined. In 
the change detection of heterogenous images, the classes of 
each image source can be defined in the distinct frames of 
discernment because the sets of classes can be different form 
one image to another. To deal with this problem, we propose to 
combine the sources of evidence in the M-D frame constructed 
by the Cartesian product of the distinct 1-D frames. By 
doing this, we exploit more efficiently the information in the 
fusion process, and thus we can estimate more precisely the 
changes between the images, taking into account the integrity 
constraints (if any) of the model of the M-D frame. 

Let us consider n sources of heterogeneous images to 
be fused, and let us assume that the classifications of each 
image are respectively defined over the frames 0i, ... , 0„. 
The bba’s obtained from each image at the same region are 
combined as m(-) = [m\ ® m 2 0 • • • ©m n ](-), where ® denotes 
the generic fusion operator. Before applying the fusion of 
sources, the original bba ra z (.) associated with the i- th image 
(for i = 1,2, ... ,n) needs to be extended to joint states 
representation in the M-D frame using the vacuous extension 
principle [21] as follows: 

rrii(A) = m z (0i, . . . , 0 z _i, A, 0 /+ i, . . . , 0„). (5) 



A. Combination Rule in the Free Model 

In the free model, all elements (joint states) of 2^" are 
allowed to occur. Thus, there is no element that is forced to be 
impossible. 3 At first, the bba’s drawn from the classifications 
done on the images are extended into the M-D frame using 
the vacuous extension following (5). Then the conjunctive 
combination rule, denoted MDER/, of the extended bba’s in 
the M-D frame is defined VA, Y t e 2^” by 

n 

MDER /; m(A)= m,(Y,) (6) 

1=1 

7iny 2 n-ny n =A 

The mass m(A) obtained in (6) can also be computed by the 
sequential fusion of the n > 2 original (one dimension) bba’s 
as follows for A = (T, y n ) e 2^" 

mi e ... (Bn (A) = m w ... m _ l (Y)m n (y n ) (7) 

where Y = (y u y 2 , ... , y n -\) e 2^' and 

j mi©... 0B _i(-) = [mi © m 2 © • • • © m n - 1](.) 

[m 101 (.) = /*!(■). 



B. Combination Rule in the Constrained Model 

In some applications, particular joint states in 2^" are known 
impossible to happen (e.g., a forest seen by a first sensor 
that appears without any roughness by a latter radar sensor). 
All these impossible joint states are therefore constrained to 
be represented by the empty set. The set of all the integrity 
constraints is denoted 0m- The mass of the empty sets forced 
by the integrity constraints has to be redistributed to the 
other focal elements of 2 4/ \ Several redistribution principles 
can be adopted. In this letter, we propose to redistribute the 
conflicting masses to the focal elements, thanks to the classical 
normalization procedure adopted in DS rule, because of its 
simplicity. 4 The combination of n bba’s, denoted MDERds, is 
defined by VA e 2^" 



MDERds : m(A) = - — — E n mtyi) (9) 

(yi,...,y n )=A l ~ l 

where y t e 2 0 * , i = 1, . . . , n, and the total conflicting mass K 
is defined by 



This method is called the vacuous extension because we do not 
use information provided by the other sources to extend m z (A) 
in the M-D frame. Such vacuous extension is very simple since 
it does not take into account the temporal correlation among 
images. The prior knowledge of the temporal correlation (if 
available, depending on the applications) can be taken into 
account in the fusion process to improve the performances of 
multitemporal classification and change detection. 

Two combination models are proposed here. The free com- 
bination model is applied if no prior knowledge about the 
impossible joint states is known. Whereas, the constrained 
combination should be used when some integrity constraints 
on the joint states must be taken into account. 



K = ^ ( 10 ) 

(yi,...,y n )e&M 

M 

The notation (yi, y 2 , • • • ,y n ) = E, means that the hypothesis 
(yu yi, • • • , y w ) is equivalent to Y in the underlying model M 
given the integrity constraints. For example, if we have the 
constraint (yi ;z , y 2 -j) e 0 m i n a 2-D framework, then the joint 
state (>’i u , y 2 -j U y 2 -k) can be simplified as (vi;n yi-j U y 2 - k ) = 
(yi;f, yi-,j) u (yi ;i , y 2 -k) = 0 U (yi;i, y 2 - k ) = (yvj, y 2 -k)- 

3 The impossible event is represented by the empty set in DST and DSmT. 

4 MDER is a general framework that can be used with different rules of 
combinations. We only propose a DS alike fusion rule for MDER here for its 
relatively low complexity, but other rules as in [13, vol. 2] can also be used. 
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(a) (b) ( C ) 

Fig. 1. ERS and SPOT images of Gloucester U.K. (a) ERS image, acquired 
on November 16, 1999 before the flood, (b) SPOT image, acquired on October 
21, 2000 during the flood, (c) Ground truth. 

All the bba’s defined with respect to a 1 -D frame can also be 
(vacuously) extended to M-D frame using ( 5 ) at first, and then 
they are combined with a proper redistribution of conflicting 
beliefs. 

IV. Application of MDER on ERS and SPOT Images 

In this section, we show the performances of MDERds for 
the change detection between a pair of heterogenous images 
(an ERS and a SPOT image) obtained before and immediately 
after a flood in Gloudcester area in U.K. The ERS and Spot 
images are shown in Fig. 1 , as well as the binary image of 
the ground truth. 

We want to focus on the application of MDER on fusion of 
the classified images, and the appropriate classification method 
can be selected according to the actual applications. The 
classification of images plays a basic role in change detections 
problem, and the more accurate classification results lead to 
the better fusion results using MDER or other approaches. A 
very recent clustering method called belief functions C-means 
(BFCM) [ 22 ] as an extension of Fuzzy C-means (FCM) under 
belief functions framework can well model the imprecision 
and uncertainty. BFCM is applied to classify the pixel values 
of each pixel of the images, and its results can be directly 
used as bba’s. For decision making, the maximum pignistic 
probability is used to select the most likely hypothesis. 

The number of clusters for ERS image despeckled by a 
refined Lee filter [ 23 ] with a 7 x 7 sliding window is given by 
^ers = 3 . The classification results are defined in the frame as 
[from lower to higher level of surface roughness in Fig. 1 (a)] 
follows: 

0 \ = Dark area 62 = Gray area 

63 = White area 0 = Ignorance. 

The SPOT image in Fig. 1 (b) was clustered into kspoT = 5 
groups of possible classes as listed below: 

co\ = Red area (covered fields) 

C02 — Dark-red area (wooded area) 

CO3 = Green area (bare soil) 
co 4 = Dark-green area (vegetation) 

C05 = Bright-green area (bare soil and wet area) 

£2 = Ignorance. 




Fig. 2. Fusion results of the pair of ERS and SPOT images: (a) Fusion results 
by MDERds- (b) Comparison of the change detections with ground truth by 
MDERds- (c) Fusion results by DSmC (Ch denotes the change occurrences), 
(d) Comparison of the change detections with ground truth by DSmC. 



The class that each cluster corresponds to is identified ac- 
cording to the clustering center. The cluster corresponds to 
the class of content whose pixel value is most close to the 
vector of this clustering center. For example, in the SPOT 
image, the related distance is the Euclidean distance in M 3 
to the spectral signature that characterizes the class center. 
The related colors refer to the pseudo-color composition Near 
Infrared, Red, Green showed as red, green, blue colors. In ERS 
image, which has been filtered, the distance to cluster refers to 
the Euclidean distance in M and the level of radiometry (from 
dark to white) is related to surface roughness (from 0\ to 63). 

The constrained model is used since we have some prior 
knowledge about this region in the image. The covered fields 
coi and wooded area (02 are linked with low roughness 0 \ , and 
the bare soils 003 are linked with high roughness 63 in the ERS 
observation. The vegetation CO4 and the wet area C05 mainly 
correspond to the gray area 62 in Fig. 1 (a). Thus, one has five 
consistent joint states (6\, co\ ), (6\, 002 ), (O3, CO3 ), ( 62 , 004), and 
(O2 , C05). We also know that the change mainly occurred around 
the flooding zone. The flood destroyed the wooded area with 
low roughness 6 \, and changed it to bare soil (03 . Therefore, 
the change occurrence mainly corresponds to the inconsistent 
joint state (61,(03). As a result, we select the five consistent 
joint states and one inconsistent joint state here. 

DSmT has already been applied for change detection of 
RS images in [ 14 ]. To compare the combination rule DSmC 
[ 13 ] with MDER D s, 5 we mapped the frame © to £2 as 
6\ = { co \ , (O2),02 = {(O4, (05}, 63 = {003} . As for the change de- 
tection, one has 61^003 = (co\ \J 002)^(03 = (co\nco3)U(co2rico3). 
So both co\ n (03 and C02 n (03 will be considered as the useful 

5 DSmC is the simple conjunctive rule, and all the possible conjunction 
between elements are allowed in this letter. 
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TABLE I 

PERFORMANCES OF THE MDER FUSION RESULTS (IN %) 





Ra 


Rm 


Rf 


K a 


DSmC 


65.52 


42.46 


34.48 


55.65 


MDERds 


81.80 


16.53 


18.20 


80.30 



conflict element representing the change occurrences. All the 
other conflict elements are constrained to be empty in the 
integrity model, and the conflicting beliefs are proportionally 
redistributed to the available elements similarly to the DS rule. 
The fusion results by different rules are shown in Fig. 2. 

As a measure of performance, we use the accuracy rate R a = 
n a /Nd , the missing rate R m = n m /N c , the false alarm rate Rf = 
rif/Nd , and the Kappa index K a . n a is the number of pixels of 
correct change detections, Nd is the total number of pixels of 
detected changes, n m is the number of pixels of nondetected 
changes, N c is the total number of pixels of changes in ground 
truth, and rif is the number of pixels of false alarms. The 
performances obtained by MDER ds and DSmC are listed in 
Table I. 

As we can see, MDER ds provides much better fusion 
results than DSmC because DSmC produces more false alarms 
than MDERds since the intersection element cannot precisely 
represent the change occurrences. The Kappa index can com- 
prehensively reflects the quality of the classification results 
with respect to the change detections. So MDERds has better 
quantitative performance than DSmC according to the K a 
value. However, a fine analysis of results indicates that some 
missed detections and false alarms have also occurred. Our 
results show that gray and white areas along the river remain 
mainly consistent with green areas between the two images. So 
they are not considered as change occurrences, which mainly 
leads to the miss detections. The reason for the false alarms 
mainly lies in changes in some small areas that resemble to 
some changes in the flood area. The number of the false 
alarms can be reduced if additional prior information about 
the location of the flood is taken into account. 

V. Conclusion and Perspectives 

A MDER approach was proposed for the fusion of the 
heterogeneous remote sensing images. The multidimensional 
elements in MDER could well represent the joint states of 
different images, which was useful for change detection. The 
belief function, plausibility function, and pignistic probability 
in MDER were defined similarly as in DST. The free model 
of MDER was designed for the combination of sources of 
evidence, in cases, where no prior knowledge about the joint 
states was available. If some constraints on the impossible joint 
states were acquired, the constrained model has to be adopted 
to get better fusion results with less computational complexity. 
MDERds rule was presented in the constrained model as a 
direct extension of DS rule in MDER. The capacity of MDER 
to detect and to classify change occurrences was also presented 
by a pair of real images (i.e., ERS and SPOT). MDER could 
also be used for the heterogeneous images mapping appli- 
cations, and the experiments were under progress, as well as 
investigations on the use of other fusion rules based on the pro- 
portional conflict redistribution principle proposed in DSmT. 
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